1. Field
Certain aspects of the present disclosure generally relate to distributed computing systems and, more particularly, to a method and apparatus for time management and scheduling for synchronous processing on a cluster of processing nodes.
2. Background
An artificial neural network is a mathematical or computational model composed of an interconnected group of artificial neurons (i.e., neuron models). Artificial neural networks may be derived from (or at least loosely based on) the structure and/or function of biological neural networks, such as those found in the human brain. Because artificial neural networks can infer a function from observations, such networks are particularly useful in applications where the complexity of the task or data makes designing this function by hand impractical.
One type artificial neural network is the spiking neural network, which incorporates the concept of time into its operating model, as well as neuronal and synaptic state, thereby increasing the level of realism in this type of neural simulation. Spiking neural networks are based on the concept that neurons fire only when a membrane potential reaches a threshold. When a neuron fires, it generates a spike that travels to other neurons which, in turn, raise or lower their membrane potentials based on this received spike.
A neural processing system typically includes a plurality of processing nodes with each processing node connected to one or more other processing nodes. In certain aspects, a large neural simulation may need to be distributed across multiple nodes. A distributed simulation may require nodes to keep in sync with each other and avoid running too fast or too slow in order to prevent data loss. Further, nodes should be able to communicate in the same time scale.
In certain aspects, each node may increment its time on receiving an indication from every other node in the processing system that a current processing step is completed, and then start a subsequent processing step. Thus, the slowest node typically governs the time increments, and thus, processing by other nodes.